A SENSORY-MOTOR LINGUISTIC FRAMEWORK FOR HUMAN ACTIVITY UNDERSTANDING

We empirically discovered that the space of human actions has a linguistic structure. This is a sensory-motor space consisting of the evolution of joint angles of the human body in movement. The space of human activity has its own phonemes, morphemes, and sentences. We present a Human Activity Language (HAL) for symbolic non-arbitrary representation of sensory and motor information of human activity. This language was learned from large amounts of motion capture data. Kinetology, the phonology of human movement, finds basic primitives for human motion (segmentation) and associates them with symbols (symbolization). This way, kinetology provides a symbolic representation for human movement that allows synthesis, analysis, and symbolic manipulation. We introduce a kinetological system and propose five basic principles on which such a system should be based: compactness, view-invariance, reproducibility, selectivity, and reconstructivity. We demonstrate the kinetological properties of our sensory-motor primitives. Further evaluation is accomplished with experiments on compression and decompression of motion data. The morphology of a human action relates to the inference of essential parts of movement (morpho-kinetology) and its structure (morpho-syntax). To learn morphemes and their structure, we present a grammatical inference methodology and introduce a parallel learning algorithm to induce a grammar system representing a single action. The algorithm infers components of the grammar system as a subset of essential actuators, a CFG grammar for the language of each component representing the motion pattern performed in a single actuator, and synchronization rules modeling coordination among actuators. The syntax of human activities involves the construction of sentences using action morphemes. A sentence may range from a single action morpheme (nuclear syntax) to a sequence of sets of morphemes. A single morpheme is decomposed into analogs of lexical categories: nouns, adjectives, verbs, and adverbs. The sets of morphemes represent simultaneous actions (parallel syntax) and a sequence of movements is related to the concatenation of activities (sequential syntax). We demonstrate this linguistic framework on real motion capture data from a large scale database containing around 200 different actions corresponding to English verbs associated with voluntary meaningful observable movement

An Optimal Time-Space Algorithm for Dense Stereo Matching

An original survey addressing time-space complexity covers several stereo matching algorithms and running time experiments are reported. Taking the point of view that good reconstruction needs to be solved in feedback loops, we then present a new dense stereo matching based on a path computation in disparity space. A procedure which improves disparity maps is also introduced as a post-processing step for any technique solving a dense stereo matching problem. Compared to other algorithms, our algorithm has optimal time-space complexity. The algorithm is faster than "real-time" techniques while producing comparable results. The correctness of our algorithm is demonstrated by experiments in real and synthetic benchmark data

Learning Parallel Grammar Systems for a Human Activity Language

We have empirically discovered that the space of human actions has a linguistic structure. This is a sensory-motor space consisting of the evolution of the joint angles of the human body in movement. The space of human activity has its own phonemes, morphemes, and sentences. In kinetology, the phonology of human movement, we define atomic segments (kinetemes) that are used to compose human activity. In this paper, we present a morphological representation that explicitly contains the subset of actuators responsible for the activity, the synchronization rules modeling coordination among these actuators, and the motion pattern performed by each participating actuator. We model a human action with a novel formal grammar system, named Parallel Synchronous Grammar System (PSGS), adapted from Parallel Communicating Grammar Systems (PCGS). We propose a heuristic PArallel Learning (PAL) algorithm for the automatic inference of a PSGS. Our algorithm is used in the learning of human activity. Instead of a sequence of sentences, the input is a single string for each actuator in the body. The algorithm infers the components of the grammar system as a subset of actuators, a CFG grammar for the language of each component, and synchronization rules. Our framework is evaluated with synthetic data and real motion data from a large scale motion capture database containing around 200 different actions corresponding to verbs associated with voluntary observable movement. On synthetic data, our algorithm achieves 100% success rate with a noise level up to 7%

A Language for Human Action

Human-centered computing (HCC) is centered on humans and what they do, i.e. human actions. Thus, developing an infrastructure for HCC requires understanding human action, at some level of detail. We need to be able to talk about actions, synthesize actions, recognize actions, manipulate actions, imitate actions, imagine and predict actions. How could we achieve this in a principled fashion? This paper proposes that the space of human actions has a linguistic structure. This is a sensory-motor space consisting of the evolution of the joint angles of the human body in movement. The space of human activity has its own phonemes, morphemes, and sentences. We present a Human Activity Language (HAL) for symbolic non-arbitrary representation of visual and motor information. In phonology, we define atomic segments (kinetemes) that are used to compose human activity. In morphology, we propose parallel learning to incorporate associative learning into a language inference approach. Parallel learning solves the problem of overgeneralization and is effective in identifying the active joints and motion patterns in a particular action. In syntax, we point out some of the basic constraints for sentence formation. Finally, we demonstrate this linguistic framework on a praxicon of 200 human actions (motion capture data obtained by a suit) and we discuss the implications of HAL on HCC

Problemas de proximidade e de caminhos minimos em superficies poliedricas

Orientador: Pedro Jussieu de RezendeDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Planejamento de Caminho Mínimo é a área em Geometria Computacional que se preocupa com a determinação dos menores caminhos possíveis de um ponto a outro em um dado ambiente. Abordamos um problema (PGAD) de caminhos mínimos direcional que procura minimizar o trabalho total realizado para se mover um corpo sobre uma superfície poliédrica com coeficientes de atrito e inclinação constantes em cada face. Sua importância se deve ao fato de que este problema generaliza vários outros. Realizamos a caracterização de caminho geodésico e mínimo segundo as restrições do problema, identificando o critério de otimalidade local correspondente. Para isso, demonstramos a convexidade estrita da função distância geodésica atritada direcional (FGAD) utilizando a teoria de funções convexas. Desenvolvemos um algoritmo, baseado na metodologia Dijkstra contínuo, para resolver o problema PGAD. O algoritmo possui algumas particularidades relacionadas ao caráter direcional devido à função distância FG AD depender da direção de movimento e à caracterização de caminhos geodésicos. Realizamos a prova de corretude e a análise de complexidade do algoritmo proposto. Além disso, identificamos alguns detalhes omitidos em algoritmos Dijkstra contínuo encontrados na literatura e os completamos. Estendemos o problema PGAD obtendo um algoritmo para construir um diagrama de Voronoi (VGAD) de caminhos mínimos sobre uma superfície poliédrica segundo a função distância FGAD. Reduzimos algumas generalizações de problemas de proximidade ao da construção deste diagrama e, dessa forma, o diagrama VGAD resolve estes problemas. Implementamos um módulo externo ao programa Geomview para visualizar uma árvore de caminhos mínimos e um diagrama de Voronoi em superfície poliédrica para o problema da geodésica discreta (PGD) que é um caso especial do PGAD.Abstract: Shortest Path Planning is the field of Computational Geometry that concerns the determination of feasible shortest paths from a point to another in a given environment. We deal with a directed shortest path problem (DFGP) that minimizes the total work spent to move a body on a polyhedral surface with constant friction coefficient and constant slope in each face. Its importance is due to the fact that it generalizes several others. In order to characterize geodesic paths and shortest paths according to the constraints of the problem, we identify the corresponding local optimality criterion and we demonstrate the strict convexity of the directed frictioned geodesic distance function (DFGF) using convex function theory. We develop an algorithm, based on the continuous Dijkstra methodology, to solve the DFGP problem. The algorithm contains some details related to the directed nature of the paths which is due to the distance function DFGF being dependent on the direction of motion and to the characterization of the geodesic paths. We prove the correctness of the proposed algorithm and analyze its complexity. Furthermore, we identify some details omitted in a few continuous Dijkstra algorithms found in the literature and fill them in. We extend the DFGP problem and obtain an algorithm to construct a shortest path Voronoi diagram (DFGV) on a polyhedral surface according to the distance function DFGF. We reduce some generalizations of proximity problems to the construction of this diagram and, therefore, the DFGV diagram solves these proximity problems. We implement an external module to the Geomview program to be able to visualize a shortest path tree and a Voronoi diagram on polyhedral surfaces to the discreet geodesic problem (DGP) that is a special case of DFGF.MestradoMestre em Ciência da Computaçã

Understanding visuomotor primitives for motion synthesis and analysis

The problem addressed in this paper concerns the representation of human movement in terms of atomic visuo-motor primitives considering both generation and perception of movement. We introduce the concept of kinetology, the phonology of human movement, and five principles on which such a system should be based: compactness, view-invariance, reproducibility, selectivity, and reconstructivity. We propose visuo-motor primitives and demonstrate their kinetological properties. Further evaluation is accomplished with experiments on compression and decompression. Our long-term goal is to demonstrate that action has a space characterized by a visuo-motor language